1. Field
The present invention relates generally to databases, particularly to improving performance and scalability of in memory databases with a distributed cache-based database system.
2. Background
The traditional outlook towards managing transactional business data is based on a persistent transaction data processing model: a debit from one account and a credit to another account, once acknowledged to an end user, must necessarily be reflected in the underlying accounts, even if the underlying computing system suffers an immediate outage at the point of transaction completion. When the system restarts, the debit account as well as the credit account must correctly reflect their state as they existed at the precise moment the transaction was successfully acknowledged to the end user. That is, these data changes need to be durable.
Transaction processing data management systems are built around providing such guarantees. These online transaction processing (OLTP) systems are designed around ACID properties: Atomic, Consistent, Isolated and Durable. ACID properties ensure that business transactions are fully reliable and accurately reflected in the underlying system, even when hundreds of transactions are processed simultaneously in extreme OLTP environments and under unpredictable system shutdown, startup, and failure scenarios.
High-performance enterprise applications often operate in extreme OLTP environments and encounter performance bottlenecks and scalability problems when trying to access data stored in a database. Traditionally, databases have used disk-based storage devices to store data. However, disk access can be very slow. To improve database performance, main memory has been used as a data buffer or cache for data stored on disk. To further improve performance and resolve scalability issues, particularly for large enterprise applications, traditional databases have employed distributed caching systems, which combine the scalability of distributed systems with the reduced access latency of main memory.
However, existing distributed caching systems pose significant challenges for database application developers and database administrators. For example, database administrators must determine how best to design and organize a distributed caching system in order to avoid database efficiency and application performance problems associated with large amounts of data movement required during cache startup. This involves a detailed understanding of not only which data should be cached, but also how and when such data is accessed. As information related to an enterprise database environment is often unavailable or difficult to acquire, it may be extremely difficult for database administrators to know which cache data needs to be populated during a cache startup or which data needs to be saved during a cache shutdown. It is also difficult to move and rebalance large amounts of data among nodes in a distributed caching system without disrupting the performance of the enterprise database environment.
Accordingly, what is needed are systems, methods, and computer program products which provide non-disruptive startup, node rebalancing, and shutdown in a distributed caching system. What is further desired are systems, methods, and computer program products to allow database administrators to move large amounts of partitioned data within a distributed caching system without disrupting the performance of applications in an enterprise database environment.